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DECLARATION 



The undersigned, Jan McLin Clayberg, having an office at 
5316 Little Palls Road, Arlington, VA 22207-1522, hereby states 
that she is well acquainted with both the English and German 
languages and that the attached is a true translation to the best 
of her knowledge and ability of International Patent Application 
PCT/DE 2005/050991 of NIEM, W., et al • , entitled "SECURITY SYSTEM 
AND METHOD FOR OPERATING IT". 

The undersigned further declares that the above statement 
is true; and further, that this statement was made with the 
knowledge that willful false statements and the like so made are 
punishable by fine or imprisonment, or both, under Section 1001 
of Title 18 of the United States Code and that such willful false 
statements may jeopardize the validity of the application or 
document or any patent resulting therefrom. 
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SECURITY SYSTEM AND METHOD gffffi^fflg^ O 4 * 1 JUL 2006 

Prior Art 

5 The invention relates to a security system as generically defined by the 
preamble to claim 1 and to a method for operating the security system as 
generically defined by the preamble to claim 4. Such security systems are 
often equipped with stationary cameras. Detecting movement or change 
with stationary cameras is a basic function of systems for radio-based 

10 security technology. The products range from surveillance cameras that 
issue alarms to digital video recorders which allow a content-based search 
for moving objects. Detecting moving objects is also a basic function in 
analyzing image sequences and is thus an important component for 
instance of systems for man-machine interaction (such as gesture control) 

15 or biometric systems (for instance, face detection with ensuing face 
recognition). 

Both the systems described in the scientific literature and those on the 
market for detecting moving objects implicitly or explicitly use a camera 
20 sensor model which assumes that the time-related noise in a pixel ("pixel 
noise") is independent of the gray value. Such systems are described for 
instance in the following places in the literature: 

A. Elgammal, D. Harwood, L. Davis, "Non-Parametric Model for 
25 Background Subtraction", FRAME-RATE workshop, 1999. 

K. Toyama, J. Krumm, B. Brumitt and B. Meyers, "Wallflower: 
Principles and Practice of Background Maintenance", ICCV 1999. 



-1- 



A. Elgammal, R. Duraiswami, D. Harwood, L. Davis, "Background and 
Foreground Modeling Using Nonparametric Kernel Density Estimation for 
Video Surveillance", Proc. of the IEEE, Vol. 90, No. 7, July 2002, pp. 1151- 
1163. 

5 

M. Meyer, M. Hotter, T. Ohmacht, "A New System for Video-Based 
Detection of Moving Objects and its Integration into Digital Networks", in 
Proceedings of IEEE Intern. Conference on Security Technology, 
Lexington, USA, 1996, pp. 105-110. 

10 

T. Aach, A. Kaup, R. Mester, "Change Detection in Image Sequences 
using Gibbs Random Fields: A Bayesian Approach", Proceedings Intern. 
Workshop on Intelligent Signal Processing and Communication Systems, 
Sendai, Japan, Oct. 1993, pp. 56-61. 

15 

The assumption of gray value independent of the pixel noise in the prior 
art is clearly incorrect, especially in the widely used sensors employing 
CCD technology. Instead, in reality, an increase in the noise variance of a 
pixel with the corresponding gray value must be expected. The usual 

20 simplifying assumption in the industry of pixel noise independent of the gray 
value has an adverse effect on the performance of the entire security 
system. For instance, in conventional security systems this assumption 
means that there must be a fixed decision threshold relating to the gray 
value, if a distinction is to be made between a gray value change because 

25 of sensor noise and a gray value change because a moving object has 

been detected. However, since the noise behavior in most image sensors is 
gray value-dependent, this means that the aforementioned decision 
threshold set is too sensitive for bright image regions and too insensitive for 
dark image regions. 
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Advantages of the Invention 

The security system of the invention having the characteristics of claim 
5 1 , conversely, leads to a substantial improvement over conventional 
security systems. Because the decision threshold is designed to be gray 
value-dependent, the security system can be better adapted to both bright 
and dark image regions. This leads to substantially more-enhanced 
sensitivity of the security system. Because a gray value-dependent noise 

10 behavior is taken into account in defining the decision threshold, it is now 
possible even to detect dark objects in dark image regions, without 
generating mistaken detections caused by pixel noise in bright image 
regions. Advantageously, the detection precision is thus increased without 
causing an increase in the rate of mistaken detections. The lowest possible 

15 rate of mistaken detections, however, is of especially great significance in 
security technology. 

Drawings 

20 The invention is described in further detail below in conjunction with the 
drawings. 

Fig. 1 , in a graph, shows the variance of the noise value as a function of 
the gray value g; 

25 

Fig. 2 shows the display of the gray value-dependent noise of a CCD 
camera; 

Fig. 3 shows one image of an image sequence that includes a plurality 
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of images; 



Fig. 4 shows one exemplary embodiment of the security system of the 
invention; 

5 

Fig. 5 is a flow chart; and 

Fig. 6 is a further flow chart. 

1 0 Description of the Exemplary Embodiments 

The assumption of gray value independent of the pixel noise in the prior 
art is clearly incorrect, especially in the widely used sensors employing 
CCD technology. Instead, in reality, an increase in the noise variance of a 

15 pixel with the corresponding gray value must be expected. The usual 

simplifying assumption in the industry of pixel noise independent of the gray 
value has an adverse effect on the performance of the entire security 
system. For instance, in conventional security systems this assumption 
means that a fixed decision threshold relating to the gray value is 

20 understood if a distinction is to be made between a gray value change 

because of sensor noise and a gray value change because a moving object 
has been detected. However, since the noise behavior in most image 
sensors is gray value-dependent, this means that the aforementioned 
decision threshold set is too sensitive for bright image regions and too 

25 insensitive for dark image regions. This situation is illustrated in Fig. 1 . In 
the graph shown in Fig. 1, the variance of the noise value is plotted as a 
function of the gray value g of the kind measured for a typical CCD camera. 
It can be seen from the measured values that the noise variance increases 
essentially linearly with the gray value g. This effect is displayed in Fig. 2. 
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Fig. 2, gray-value-coded, shows the variance of the pixel noise that would 
be determined by evaluating a sequence (of approximately 30 images) from 
a static scene. Bright pixels represent a high noise variance, and dark 
pixels represent a low noise variance. The image sequence itself shows the 
5 same picture content in all the images. A single image in this sequence is 
shown in Fig. 3. 

If the noise variance were gray value-independent, then Fig. 2 would 
have to represent an unstructured gray area. As can be seen from this 

10 drawing, however, the noise variance depends on the gray value of the 

pixels in the image sequence. As a consequence of this dependency, bright 
objects in the image sequence (see Fig. 3) also appear bright in Fig. 2 (high 
noise variance). The dark areas in Fig. 2 result from overloading effects in 
the original sequence, or in other words sticking of pixels at a fixed gray 

15 value. 

An optimal decision threshold would be gray value-dependent and 
would correspond in its qualitative course to the course of the curve marked 
"noise variance over the gray value"; that is, for dark image regions, the 
20 threshold would be lower than for bright pixels. In the case of a sensor with 
a linear course of this curve (see also Fig. 1), the decision threshold would 
also have to exhibit a linear behavior over the gray value. 

The security system of the invention having the characteristics of claim 
25 1 conversely leads to a substantial improvement over conventional security 
systems. The invention is based on the recognition that substantially better 
results can be attained if the decision threshold is adapted adaptively. 
Because the decision threshold is now designed to be gray value- 
dependent, the security system can be better adapted to both bright and 
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dark image regions. This leads to substantially more-enhanced sensitivity of 
the security system. Because a gray value-dependent noise behavior is 
taken into account in defining the decision threshold, it is now possible even 
to detect dark objects in dark image regions, without generating mistaken 
5 detections caused by pixel noise in bright image regions. Advantageously, 
the detection precision is thus increased without causing an increase in the 
rate of mistaken detections. The lowest possible rate of mistaken 
detections, however, is of especially great significance in security 
technology. 

10 

One exemplary embodiment of the security system 100 according to the 
invention and its operating phases will be described below, in conjunction 
with Fig. 4 and the flow charts shown in Figs. 5 and 6. The security system 
includes two subsystems 101 and 102. The first subsystem is substantially 
15 in operation during a first operating phase, while the second subsystem is 
active during a second operating phase. 

The security system 100 includes at least one camera 3 with an image 
sensor 4, and this camera is associated with both subsystems 101, 102 and 

20 is active in both operating phases of the security system 100. The security 
system 100 also includes a plurality of function modules 1, 6, 8, 9, 15, 
which are linked in terms of circuitry or at least functionally to the camera 3. 
The subsystem 101 , besides the camera 3, includes a function module 1 
with a light source. The brightness of this light source is controllable as a 

25 function of time. The subsystem 101 further includes a function module 6 
for displaying a digital image sequence from the output signals of the image 
sensor of the camera 3. Finally, the subsystem 101 includes a function 
module 8 for displaying the noise variance as a function of the gray value 
from the digital image sequence. The subsystem 102, besides the camera 3 
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with the image sensor 4, includes a function module 13, which in turn 
comprises two function modules 13a, 13b. The function module 13a serves 
to calculate or estimate the gray value variance from the output signals of 
the image sensor 4 of the camera 3. The function module 13b makes a 
5 comparison with a threshold value possible. The security system 100 
further includes a memory 9, to which both subsystems have access. 

In this security system 100, two operating phases can be distinguished, 
which will now be discussed in succession. In the first operating phase, 

10 initialization of the security system 100 is done in the offline mode (flow 

chart in Fig. 5). In a second operating phase, the security system 100 takes 
up its security task in the online mode (flow chart in Fig. 6). In the first 
operating phase of the security system 100, it is essentially the subregion 
101 that is active, while in the second operating phase, it is the subregion 

15 1 02 that is active. Below, first, the first operating phase of the security 

system 100, which is shown in the subregion 101, will be explained. During 
the initializing phase, the noise variance is determined by a measuring 
system as a function of a gray value of the image sensor 4 located in the 
camera 3. For this purpose, a light source is furnished by the function 

20 module 1 and makes it possible to stimulate the camera 3 having the image 
sensor 4. The brightness of the light source in the function module 1 is 
increased in small increments as a function of time, and after each increase 
it is kept constant for a predeterminable length of time. This results in the 
stairstep curve, represented in the function module, for the course of the 

25 brightness. After the pickup (step 50 in Fig. 5) of the light signals generated 
by the function module 1 by means of the camera 3 and digitization (step 51 
in Fig. 5) of the output signals of the image sensor 4 of the camera 3, a 
digital image sequence is then present in the function module 6, after the 
complete execution of the stimulation of the camera 3. The evaluation of 
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this image sequence (step 52 in Fig. 5) in the function module 8 leads to a 
characteristic curve that represents the noise variance of the image sensor 
4 as a function of the gray value g. This characteristic curve is stored in a 
memory 9 (step 53 in Fig. 5), with the image sensor 4 used being indicated, 
5 and is now available to the security system 100 for ongoing operation. The 
qualitative form of this curve, in particular, forms an important basis for 
reliable detection of an object in the area to be monitored. The shape of the 
characteristic curve is used to adapt the characteristic of the gray-value- 
dependent decision threshold adaptively accordingly. For cameras with 
10 automatic camera regulation, in an advantageous further feature of the 
invention, the memory 9 is additionally supplied with data which represent 
the dependency of the noise variance on the gray value for the various 
camera parameters. 

15 The second operating phase of the security system 100 is schematically 
shown in the subregion 102 of the drawing. The system operates in ongoing 
operation as follows. A natural scene (recording field 10) that corresponds 
to the area being monitored is examined in terms of the scene contents for 
whether a change in pixels of the images taken by the camera 3 is 

20 occurring because of sensor noise, or because of a moving object. Once 
the natural scene 10 has been recorded (step 60 in Fig. 6) and after 
ensuing digitization (step 61 in Fig. 6), a digital image sequence is 
available, which is now analyzed by a function module 13 for the presence 
of moving objects, such as people, vehicles, or other things. To that end, in 

25 a function module 13a, the gray value variance is first calculated or 
estimated (step 62 in Fig. 6). The methods described in the literature 
calculate or estimate, from chronologically successive images, first the gray 
value variance for each pixel, which is composed additively of one 
component that can be ascribed to pixel noise and one component that can 
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be ascribed to motion in the scene. If there is a significant change in a pixel 
that does not fit the noise model of the pixel, then on the basis of a 
threshold value decision in the function module 13b it is decided (step 63 in 
Fig. 6) that a moving object is present. The threshold value in the method of 
5 the invention is not predetermined identically for all gray values, as has 
conventionally been done until now. but instead is adapted adaptively from 
the sensor characteristic curve ascertained in the function module 8 and 
stored in the memory 9. 

10 In a further step (step 64 in Fig. 6), a mask is generated in order to 
mark a motion in the area being monitored as a segmentation result 15. 
The person in the right foreground is clearly apparent. 

As described above, it is useful during the initializing phase for data 
1 5 about the operating state of the sensor 4 as well as camera parameters, 
such as the amplification, to be forwarded to the function module 8 that 
determines the noise curve, so that possible changes in the gray-value- 
dependent noise characteristic can be taken into account. For instance, it is 
possible that the amplifier noise of the image sensor 4, in low light 
20 conditions, will cover the noise in a picture element and thus change the 
gray-value-dependent of the noise. To make it possible to utilize this option 
even during ongoing operation, the operating state of the image sensor 4 
must be forwarded to the function module 13b for the threshold value 
decision. 

25 

The essential nucleus of the invention is thus the use of an adaptive, 
gray-value-dependent threshold value decision for detecting objects. By this 
provision, the performance and precision of recognition by such a security 
system is enhanced substantially. The threshold values are expediently 
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measured in advance in the form of characteristic curves as a function of 
the gray value and of the camera parameters and are stored in a memory 9. 
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